Mirror Descent Search and Acceleration

In recent years, attention has been focused on the relationship between black box optimization and reinforcement learning. Black box optimization is a framework for the problem of finding the input that optimizes the output represented by an unknown function. Reinforcement learning, by contrast, is a framework for finding a policy to optimize the expected cumulative reward from trial and error. In this research, we propose a reinforcement learning algorithm based on the mirror descent method, which is general optimization algorithm. The proposed method is called Mirror Descent Search. The contribution of this research is roughly twofold. First, an extension method for mirror descent can be applied to reinforcement learning and such a method is here considered. Second, the relationship between existing reinforcement learning algorithms is clarified. Based on these, we propose Mirror Descent Search and derivative methods. The experimental results show that learning with the proposed method progresses faster.